Selection of a mipmap level

ABSTRACT

A computer graphics includes a texture memory ( 134 ) storing texture maps in a mipmap structure, texels in a texture map being specified by a pair of u and v coordinates. A rasterizer ( 120 ) determines for a texel (u, v) corresponding initial 4D mipmap levels (mml u , mml v ) and a magnification factor representing a magnification that occurs when the texel is mapped to a corresponding pixel position on the display. It then determines final 4D mipmap levels in dependence on the determined initial 4D mipmap levels mml u , mml v , and the magnification factor. A texture space resampler ( 132 ) obtains texture data from a texture map identified by the pair off final 4D mipmap levels. A texture mapper ( 140 ) maps the obtained texture data to corresponding pixel data defining the display image.

FIELD OF THE INVENTION

The invention relates to a system and a method of computer graphicsprocessing.

BACKGROUND OF THE INVENTION

An important element in rendering 3D graphics is texture mapping.Mapping textures onto surfaces of computer-generated objects is atechnique which greatly improves the realism of their appearance. Thetexture is typically a 2D picture, such as a photograph or computergenerated image. For example, (part of) a 2D image of a wall may beprojected on a 3D representation of a wall in a computer game. Most 3Dobjects cover only a small part of the screen, often resulting inminification of the texture map (which is of sufficient resolution toalso provide a reasonable appearance when viewed up close). Often,during texture mapping the 2D picture has to be minified considerably,for example if the wall is far removed. In principle, texture mappingcould then be performed by significantly downscaling the original image.To reduce the bandwidth required for reading the high resolution 2Dpicture, a pre-processing step is often performed in which severaldownscaled versions of the 2D picture are created. During texturemapping, the part of only the smaller downscaled picture which matchesbest in resolution with the screen image is read and mapped to thescreen. The original 2D picture along with its downscaled versions iscalled a mipmap. Texture mapping as well as mipmaps are particularlydescribed in “Survey of Texture Mapping Paul S. Heckbert, IEEE ComputerGraphics and Applications, November 1986, pp. 56-67 and in U.S. Pat. No.6,236,405 B1. In a mipmap, the original image is denoted as level 0. Inlevel 1, each entry holds an averaged value of, for example, 2×2 texels.As used herein the term “texel” (texture element) refers to a pictureelement (pixel) of the texture. This can be continued until thetop-level is reached, which has only one entry holding the average colorof the entire texture. Thus, in a square mipmap, level n has one fourththe size of level n-1. In the present specification, the term “texture”is used as a synonym for any image or structure to be mapped onto anobject.

Several types of mipmaps are known, varying in which downscaled imagesare stored. In a 3D mipmap, both directions are downscaled by the samefactors, while in a 4D mipmap the original image is downscaledindependently in both dimensions. Compared to the 3D mipmap, the 4Dmipmap arrangement requires a lot of memory to store. Computer graphicsprograms, such as games, therefore, often use the 3D mipmap structure.The non-prepublished European patent application with attorney docketnumber PHNL010924EPP filed as IB02/05468 describes a method forgenerating a 4D mipmap on the fly from a 3D mipmap. This enables highquality rendering also in combination with programs that do not supply a4D mipmap to the graphics system.

In general, there are several methods known for mapping the (mipmapped)image onto the screen grid. Most conventional computer graphics systemsuse a so-called inverse texture mapping approach. In this approach,pixels of the screen are processed sequentially and for each pixel,during a rasterization process, a projection of the screen pixel on thetexture (resulting in a pixel's “footprint”) is determined and anaverage value which best approximates the correct pixel color iscomputed, usually in the form of a weighted average. An alternativeapproach is the so-called forward texture mapping method. This methodworks by traversing texels in the coordinate system defined by thetexture map. The texel colors are then splatted to the screen pixels,using resamplers commonly used for video scaling.

A 2D or 3D object to be rendered is typically modeled using primitives(usually triangles). A vertex shader of the graphics system receives thevertices of a primitive as input and uses a vertex shading program tochange or add attributes for each of these vertices. In a forwardmapping system, a rasterizer then traverses the primitive in texturespace while interpolating these attributes. The rasterizer acceptsvertex coordinates which define vertices of triangles. The rasterizercomputes texture coordinates (u, v) for each texel to be projected tothe triangle. For each grid position (u, v) of the texture visited, thetexel shader calculates from these attributes the local color of thesurface of the primitive. These surface colors are then resampled toscreen pixel locations by the screen space resampler. In a two-passforward mapping approach, mapping of a 2D image is decomposed in two 1Dmappings. First, the image is mapped in one direction, typically thescan line direction, i.e. in horizontal direction, then in the otherdirection. The first pass then processes complete scan lines andtherefore the vertical mipmap level in the texture space rasterizer isfixed for the whole primitive (triangle) that is being rendered. Therasterizer indicates for a triangle the maximum amount of detailrequired, based on information for the vertices. To be able to stillvary the horizontal mipmap level, the 4D mipmap arrangement is used. Itis also possible to use a one-pass 2D filter and vary both thehorizontal and vertical mipmap level. The supplied information includesa horizontal mipmap level mml_(u) and a vertical mipmap level mml_(v).The rasterizer processes the texels along the grid determined by thesetwo 4D mipmap level values. A texture space resampler resamples datafrom the identified 4D mipmap to the texel position.

It is desired to make a well-considered choice of the mipmap to use.Choosing a too detailed mipmap generates too much memory bandwidth.Choosing a too coarse mipmap generates an image of too low quality. Inparticular, for a 2-pass forward texture mapping graphics system wherethe vertical mipmap level is fixed for the entire triangle on the basisof the maximum amount of detail required anywhere in the triangle, therisk of choosing a too detailed level is high.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a method and system forchoosing a mipmap level.

To meet the object of the invention, a system for rendering an image fordisplay includes:

a texture memory for storing texture maps in a mipmap structure; texelsin a texture map being specified by a pair of u and v coordinates;

a rasterizer operative to, for a texel (u, v),

-   -   determine corresponding initial 4D mipmap levels (mml_(u),        mml_(v));    -   determine a magnification factor representing a magnification        that occurs when the texel is mapped to a corresponding pixel        position on the display; and    -   determine corresponding final 4D mipmap levels in dependence on        the determined initial 4D mipmap levels mml_(u), mml_(v), and        the magnification factor; and

a texture space resampler for obtaining texture data from a texture mapidentified by the pair of final 4D mipmap levels;

a texture mapper for mapping the obtained texture data to correspondingpixel data defining the display image.

The inventor has realized that, although the rasterizer operates on agrid determined by the initial 4D mipmap levels (mml_(u), mml_(v)) (in away that corresponds to the maximum amount of detail needed somewhere inthe triangle), these values (as conventionally delivered by therasterizer to the texture space resampler) may not be the ideal measurefor choosing the texture from the mipmap structure: it may indicate amuch more detailed level than is actually needed at a particular pointwithin the triangle. This is particularly the case if the triangle isviewed under a high perspective. This causes a magnification to occur inone or both of the orthogonal directions of the grid. For example, avertical magnification factor (the vertical direction is usuallyindicated by v) indicates for the current rasterization grid positionhow far apart on the screen two texels are that differ one in v value inthe vertical rasterization mipmap level mml_(v). The magnificationfactor then influences the choice of mipmap level used for the texturespace resampling. The system according to the invention may be used forinverse texture mapping systems as well as forward texture mappingsystems.

According to the measure of the dependent claim 2, the vertical 4Dmipmap level is adjusted depending on the vertical magnification. Thiscombines well with a 2-pass graphics system where the rasterizer itselfduring the first pass operates with a fixed vertical 4D mipmap level forthe entire triangle on the basis of the maximum amount of detailrequired anywhere in the triangle and a variable horizontal mipmaplevel. Such a conventional rasterizer already uses a horizontal 4Dmipmap level that varies per texel. According to the invention, therasterizer also determines per texel a final vertical 4D mipmap levelfor use by the texture space resampler by adjusting its initial vertical4D mipmap level in dependence on the vertical magnification. All otheroperations of the rasterizer are not affected and can still operate witha fixed vertical 4D mipmap level.

According to the measure of the dependent claim 3, the mipmap levelmml_(v) is lowered if the magnification factor is small. In this case, alower vertical resolution can be used. Conventionally, the vertical 4Dmipmap levels supplied by the rasterizer represent the highest level ofdetail required for a texel within a primitive. In particular if themagnification factor is small (high perspective in that direction) arelatively large minification occurs in that direction, enabling the useof a 4D mipmap with less detail in that direction.

According the measures of the dependent claim 4, the texture memorystores a 4D mipmap structure. The texture space resampler operates onthe 4D mipmap indicated by the finally determined levels (possibly witha lower resolution). The rasterizer still operates on the initial 4Dmipmap (with possibly unnecessary high detail). The resampler providesthe data to the rasterizer by reconstructing it from the finallyidentified 4D mipmap (e.g. through interpolation). In this way,bandwidth to the texture memory can be reduced at those texels where thequality does not suffer.

According the measures of the dependent claims 5, the texture memorystores a 3D mipmap structure, as is usually the case for computergraphics applications such as games. The texture space resamplergenerates the finally identified 4D mipmap on-the-fly from one of the 3Dmipmaps. The non-prepublished European patent application IB02/05468describes a method for generating a 4D mipmap on the fly from a 3Dmipmap. This enables efficient high quality rendering also incombination with programs that do not supply a 4D mipmap to the graphicssystem.

According the measures of the dependent claims 6 and 7, respectively,the 3D mipmap level can be chosen as the maximum of both 4D mipmaplevels, if priority is given to high quality rendering, or as theminimum, if priority is given to reducing memory bandwidth.

According to the measure of the dependent claim 8, also a maximumanisotropy level is taken into consideration when choosing the 3D mipmaplevel. The dependent claim 9 shows a preferred way of doing so.

To meet an object of the invention, a method of rendering an image fordisplay, includes:

storing texture maps in a mipmap structure; texels in a texture mapbeing specified by a pair of u and v coordinates;

in a rasterization operation determining, for a texel (u, v):

-   -   corresponding initial 4D mipmap levels (mml_(u), mml_(v));    -   a magnification factor representing a magnification that occurs        when the texel is mapped to a corresponding pixel position on        the display; and    -   corresponding final 4D mipmap levels in dependence on the        determined initial 4D mipmap levels mml_(u), mml_(v), and the        magnification factor;

in a texture space resampling operation, obtaining texture data for atexture map identified by the final 4D mipmap levels; and

mapping the obtained texture data to corresponding pixel data definingthe display image.

These and other aspects of the invention are apparent from and will beelucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1 shows a graphics pipeline of a forward texture mapping system;

FIG. 2 shows a graphics pipeline of an inverse texture mapping system;

FIG. 3 shows a 4D mipmap structure;

FIG. 4 illustrates reconstruction filtering;

FIG. 5 illustrates screen space pre-filtering;

FIG. 6 shows a low quality mipmap selection;

FIG. 7 shows a medium quality mipmap selection;

FIG. 8 shows visible mipmap level transitions at low quality mipmapselection; and

FIG. 9 shows a block diagram of a computer incorporating the graphicssystem according to the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Overview of the System

FIG. 1 shows an exemplary architecture of the last stages of a graphicspipeline in which the invention may be utilized. FIG. 1 shows a forwardtexture mapping system. FIG. 2 shows the last stages of a graphicspipeline of an inverse texture mapping system. In this description it isassumed that the models to be visualized are described using trianglesas primitives. Persons skilled in the art can easily apply the sametechniques to other primitives, such as other polygons or quads orcurved surfaces such as Bezier patches. Input to the pipeline areprimitives specified by its vertices by a graphics program, such as acomputer game, and the earlier stages of the graphics pipeline. Theprimitives are given in the screen space, using (x, y) coordinates aswell as the respective texel space, using (u, v) coordinates. Thepipeline includes a vertex shader 110, texture space rasterizer 120,texel shader 130 with a texture space resampler 132 and texture memory134, a screen space resampler 140 and an Edge Anti-Aliasing and HiddenSurface Removal (EAA & HSR) unit 150. The outputted pixels are stored ina frame buffer 160 for display, for example using a D/A converter, suchas a RAM DAC, to generate analogue output. If so desired also a digitalinterface, such as DVI, may be used to supply the pixel data to adisplay. The display may be of any type, including CRT, LCD, plasmadisplay. Alternatively, the rendered picture may also be used as atexture map for subsequent primitives. The functions of the graphicspipeline shown in FIGS. 1 and 2 will now be described in more detail toillustrate an exemplary system in which the invention may be used.

The vertex shader 110 of FIG. 1 and 210 of FIG. 2 receives the verticesof a triangle (primitive) as input and uses a vertex shading program tochange or add attributes for each of these vertices. The data providedby the vertex shader usually includes attributes like diffuse and/orspecular color, texture coordinates, (homogeneous) screen coordinates,and sometimes extra data like surface normals or other data required forthe shading process. The vertex shader may be a traditional Transformand Lighting unit. The attributes generated by the vertex shader areoffered to a rasterizer.

The rasterizer 220 of FIG. 2 operates in screen space, in a so-calledinverse texture mapping system (i.e. pixels from the screen space aremapped to a texture in texture space instead of projecting the textureonto pixels of the screen). Such a rasterizer uses a scanline algorithmto traverse the pixels which lie within the projection of the primitiveon the screen, by selecting the screen coordinates from the vertexattributes as driving variables for the rasterization process. Therasterizer thus traverses the primitive over a “screen grid”. Ascoordinates in the screen space are used x (for the ‘horizontal’direction) and y (for the ‘vertical’ direction).

In a preferred embodiment shown in FIG. 1, the rasterizer 120 operatesin surface space (a so-called forward texture mapping system). Theremainder of the description will focus on this preferred embodiment.Persons skilled in the art will be able to apply the principles outlinedbelow equally well to an inverse mapping system. The surface spacerasterizer traverses a parameterization of the surface of the primitive(rather than the projection on the screen), by selecting, for example,the texture coordinates (instead of screen coordinates) as the drivingvariables for the rasterization process. The rasterizer traverses theprimitive over a “surface grid”. The grid associated with a texture mapprovides such a surface grid, and is preferably used as surface grid(since obtaining texture samples on a texture grid does not requireresampling). In absence of texture maps, or when for example texturesare 1D or 3D, another grid may be chosen. As the coordinates in thetexture space are used u (for the ‘horizontal’ direction) and v (for the‘vertical’ direction). It will be appreciated that ‘horizontal’ and‘vertical’ are in this description only relative. For example, thescreen may be rotated, leaving the graphics processing unaffected butrotating the output on the screen. Since the texture grid is often usedas the surface grid, the notation “texture grid” (and “texture space”and “texel”) will be used to denote such generalized grids (andassociated spaces and samples). As the rasterizer traverses the texelpositions of the grid, all attributes that were given at each vertex areinterpolated over the grid (typically linearly, except for the screencoordinates to which a texel is projected, which are interpolatedperspectively). The attributes are then available at each texellocation, where the texel shader 130 can use them. While traversing theu and v texture coordinates of the base grid, the rasterizer alsomaintains the corresponding screen coordinates (x, y) (or vice versa foran inverse mapping system). The correspondence can be maintained bylinear interpolation of {circumflex over (x)}, ŷ and ŵ, where the{circumflex over (0)} denotes homogeneous coordinates. Such coordinatesare well-known in the art and will not be described any further here.Screen coordinates can then be calculated using the perspective divisionx={circumflex over (x)}/ŵ and y=ŷ/ŵ. Using screen space resampling basedon two 1D resampling passes, the screen y coordinate is only used formipmap determination purposes, as will be explained in more detailbelow. Using a one-pass 2D screen space resampler, they coordinate isalso used as input to the resampler. For computing the actual colors,the rasterizer may interpolate diffuse color (RGBA), specular color(RGB) and extra texture coordinate pairs (allowing for multi-texturing).Also other attributes (such as a surface normal) may be interpolated.

In the system according to the invention, the texture space rasterizertraverses the texture map on a grid corresponding to 4D mipmapping, asillustrated in FIG. 3. In a 3D mipmap, both directions are downscaled bythe same factors. A 3D mipmap is specified by the mipmap level mml. Theoriginal image is denoted as level 0. In level 1, each entry holds anaveraged value of, for example, 2×2 texels. This can be continued untilthe top-level is reached, which has only one entry holding the averagecolor of the entire texture. Thus, in a square mipmap where subsequentlevels are scaled using a factor of two in each direction, level n hasone fourth the size of level n-1. Other scaling factors may be used aswell. In a 4D mipmap the original image is downscaled independently inboth dimensions. A 4D mipmap is specified by a horizontal mipmap levelmml_(u) and a vertical mipmap level mml_(v). FIG. 3 shows a 4D mipmapgiving details of 16 mipmaps levels (0,0), (1,0), . . . , (3,3). Themipmaps levels indicated in gray (0,0), (1,1), (2,2), and (3,3) form theoriginal 3D mipmap levels 0, 1, 2, and 3, respectively. The rasterizersupplies for each texel (u,v) corresponding 4D mipmap levels (mml_(u),mml_(v)) to the texel shader 130.

The texel shader 130 computes for each texel the local surface color.The pixel shader 230 of FIG. 2 operates in an analogous way. The texelshader operates on the attributes on grid positions in the surface gridand if there are any secondary textures associated with the primitive,it uses inverse mapping with a standard texture space resampler 132 toobtain colors from these. When texture data is needed, the texture spaceresampler is used to obtain a texture sample given the texturecoordinates. These texture coordinates are generated by the texel shaderbased on the interpolated coordinates received from the rasterizer andany results from previous texture fetches (so-called dependenttexturing) and/or calculations. The texture filter operation is usuallybased on bi-linear or tri-linear interpolation of nearby texels, orcombinations of such texture probes to approximate an anisotropic(perspectively transformed) filter footprint. The 2D resamplingoperations of the texel space resampler is preferably executed in two 1Dresample passes using 1D FIR filter structures.

In a preferred embodiment, the texture memory 134 (and 234 of FIG. 2)stores texture maps in a 3D mipmap structure. In such an embodiment, thetexture space resampler 132 (and 232 of FIG. 2) is preferably arrangedto on-the-fly reconstruct a desired 4D mipmap from a 3D mipmap as willbe described in more detail below. A texture fetch then amounts to 4Dmipmap reconstruction from the 3D mipmap data stored in the texturememory 134. In the example of FIG. 3, the 4D mipmap (3,0) isreconstructed through downsampling of the 3D mipmap level 0. A fetchedtexel can be combined with interpolated diffuse and/or specular colorresulting in a color sample of the surface with associated (generallynon-integer) screen coordinates which indicate where this texture sampleis mapped to on screen. Using on-the-fly 4D mipmap reconstructionaccording to the invention, the texel shader 130 can be seen as a mipmapreconstructor. The texture space resampler obtains texture samples fromsecondary texture maps, for example, via standard bilinearinterpolation. If so desired, this can be extended to trilinear oranisotropic filtering methods, and also support for fetching of 1D and3D textures can be added (once this has been added the interpolation ofthe texture coordinates by the rasterizer also needs to be generalizedto support a possible third coordinate). Alternatively, the texturememory 134 may store texture maps in a 4D mipmap structure. In such anembodiment, the texel space resampler can simply retrieve the texel datafrom the specified 4D mipmap. As will be described below, the rasterizermay operate on a more detailed 4D mipmap. If so, the texel spaceresampler is preferably arranged to reconstruct on-the-fly the moredetailed 4D mipmap data for the rasterizer from the lesser detailed 4Dmipmap used for the resampling.

The screen space resampler 140 splats mapped texels to integer screenpositions, providing the image of the primitive on the screen. Thescreen space resampling includes the following operations:

Reconstructing the color information in the texel grid to a continuoussignal,

Mapping the continuous signal from the texture space to the screenspace,

Pre-filtering the mapped continuous signal in screen space, and

Sampling the pre-filtered signal in screen space.

It will be appreciated that also first the mapping from the texturespace to the screen space may be performed, followed by reconstructingthe mapped signal. FIG. 4 illustrates the mapping and reconstructionfiltering using a box as the footprint of the reconstruction filter.Other filters, such as higher order filters may also be used. The figureshows a grid of pixels. Each pixel is shown as a rectangle around thedimensionless location of the pixel. The solid dots illustrate thelocation of the dimensionless input texel coordinates after thetransformation (mapping). The footprints of the original texels aretaken and projected onto the screen. The size and location of thefootprints of the texel after transformation are shown as the rectangleswith dashed lines in FIG. 4.

As illustrated in FIG. 5, next, each mapped texel is then splat to (i.e.distributed over) pixels in the screen space of which the pre-filterfootprint in screen space overlaps with the reconstructed box in thescreen space (i.e. after mapping the texel to screen space). Thereconstructed box of texel 500 is shown with the highlighted rectangle510. The pre-filter footprint may extend over several pixels. The filtermay extend only horizontally, but may also have a vertical extent. Inthe example of FIG. 5, a filter is used with both a horizontal andvertical extent of three pixels, centered on the pixel to which itbelongs and covering two neighboring pixels. In this case, twelve outputpixels receive a contribution. For each of those output pixels thecontribution is determined by using the shape of their respectivepre-filter to weigh the input texel value.

The pixel fragments coming from the screen space resampler are thencombined in the Edge Anti-Aliasing and Hidden Surface Removal (EAA &HSR) unit 150 of FIG. 1 (and 250 of FIG. 2), which uses a fragmentbuffer 160 of FIG. 1 (and 260 of FIG. 2). Pixel fragments aredepth-sorted into this buffer to solve the hidden surface problem. Afterall primitives have been rendered, all visible fragments for each pixelare combined (which mostly amounts to simple summation since the screenspace resampler delivers colors already weighted by the pre-filter) andsent to the frame buffer. Edge anti-aliasing results from combining thepartial contributions generated by the screen space rasterizer near theedges, resulting in a final pixel color which can be a combination ofcolors from different primitives.

Adjusting the 4D mipmap Level

For any texture map which provided the rasterization grid, therasterizer maintains separate horizontal and vertical mipmap levels,which together form a 4D mipmap level index. In the 4D mipmap texelreconstruction process, the rasterizer provides for each rasterizationgrid position the u and v coordinates for the texel that needs to befetched, along with the 4D mipmap level (mml_(u), mml_(v)) for thattexel.

Using mipmaps where successive levels are scaled in each direction by afactor of two, typically a mipmap level is chosen that provides amagnification factor between 0.5 and 1. A magnification factor in adirection is the difference in pixels if the texel is changed by one inthat direction (e.g. increased by one). A magnification factor of 1 inboth directions for a texel area, roughly corresponds to the situationwherein the texel area has the same number as texels as there are pixelsin the area to which it is projected. A magnification factor of 0.5 inboth directions, then roughly corresponds to the situation of a texelarea with four times the number of texels as there are pixels in thearea to which it is projected. A lower magnification than 0.5 makes itbetter to use a 3D mipmap of a higher level, i.e. with lower resolution(giving sufficient resolution and requiring less bandwidth). For 4Dmipmaps, both direction-specific levels can, in principle, be chosenindependently.

Using a scaling factor of two (as is normally the case) between mipmaplevels, the mipmap level in a direction with a magnification factor s isgiven by −²log(s). Persons skilled in the art will be able to perform asimilar mipmap choice for other scaling factors and other numbering ofthe mipmap levels.

The rasterizer determines for each texel (u, v) corresponding initial 4Dmipmap levels (mml_(u), mml_(v)). In a system wherein the rasterizeruses a fixed mipmap level for the vertical direction (and/or horizontaldirection if so desired), the rasterizer determines this initial 4Dmipmap level in this direction by determining the magnification factorin this direction in the triangle vertices and choosing the largestmagnification factor. So, if for a given direction, in one vertex themagnification factor is 1 and in the other two vertices it is 0.5, itchooses 1 and calculates the mipmap level in that direction as−²log(1)=0. If the three vertices have twice 0.25 and once 0.5 asmagnification factors, this gives −²log(0.5)=1. The initial 4D mipmaplevels determine the grid on which the rasterizer operates. These levelsare not necessary the most optimal values for the texture spaceresampling for each texel within the triangle. Thus if the verticalmipmap level is fixed for operations of the rasterizer and thehorizontal mipmap level is variable, the rasterizer once per triangledetermines the vertical 4D mipmap level and per texel determines thecorresponding horizontal 4D mipmap level. According to the invention,the rasterizer also determines for each texel (u, v) a correspondingmagnification factor representing a magnification that occurs when thetexel is mapped to a corresponding pixel position on the display.According to the invention, final 4D mipmap levels are determined independence on the initial 4D mipmap levels and the magnification factor.The final 4D mipmap levels are used by the texture space resampler.Preferably, the magnification factor is only in the vertical direction,where the rasterizer itself uses a fixed vertical mipmap level pertriangle. This is particular advantageous when processing occurs in two1D scans. The first scan is then, preferably, in the display scan linedirection (‘horizontal’) giving an intermediate image. The second scanthen operates on the intermediate image in the other direction. For thefirst scan, in texture space per scan line the horizontal coordinate uis variable and the vertical coordinate v is fixed per line. Therasterizer provides a value for mml_(v) that is kept constant across thewhole triangle. The value for mml_(u) is determined per texel. In thisway, the rasterizer per texel determines the initial value mml_(u) thatalso acts as the final value fmml_(u). The initial 4D level mml_(v) thatis kept constant for operations of the rasterizer is then adjusted pertexel to the final value fmml_(v) for the texture space resampling asdescribed below.

The final vertical mipmap level fmml_(v) is determined by adjustingmml_(v) to identify a lower resolution mipmap level if the magnificationfactor is less than a predetermined threshold and maintaining thedetermined mml_(v) mipmap level otherwise. To determine themodification, the rasterizer calculates ∂y/∂v, which indicates themagnification factor for the second pass (which maps v values to yvalues). This value indicates based on the currently selectedrasterization grid position how far apart on the screen two texels arethat differ one in v value in the vertical rasterization mipmap levelmml_(v). Because of the choice of mml_(v), this is at most 1 pixelspacing, but it can be considerably less because of the variation inminification across the triangle, and because shear reduction might haverefined the rasterization grid vertically. Using scaling by a factor oftwo, the threshold is preferably ½. Persons skilled in the art will beable to determine optimal thresholds for other scaling factors. Thus, if∂y/∂v is smaller than ½, a coarser 4D mipmap level mml_(v) may be chosenthan one would assume on the initial mml_(v) (and still have aneffective screen pixel spacing between ½ and 1 for subsequent rows inthe fetched 4D mipmap level mml_(v)). More precisely, ∂y/∂v determines amipmap level difference${\Delta\quad{mml}_{v}} =^{2}{\log\left( \frac{\partial y}{\partial v} \right)}$which can be added to mml_(v) to arrive at the vertical 4D mipmap levelwhich is really needed to supply the proper amount of detail for aparticular rasterization grid location. This gives the final mipmaplevel fmml_(v)=mml_(v)+Δmml_(v). Preferably, the rasterizer in additionto the initial 4D mipmap levels provides the vertical magnificationfactor per texel. The calculation shown above to determine the finalvertical 4D mipmap level may then be done by the texture spaceresampler, but it may also be done by another unit of the system. Itwill be appreciated that above description for a preferred embodiment isfor a vertical magnification. Other systems may use the same principlein the horizontal direction or in both directions. Using the techniquedescribed above, optimal 4D mipmap levels are chosen for resamplingdespite rasterizing over a finer grid at some locations in the triangle.So although this finer rasterization costs more processing power, memorybandwidth required for fetching texture maps is not affected.

The final 4D mipmap levels can be used by the texture space resampler tofetch texture data from the specified 4D mipmap, if the texture memorystores a full 4D mipmap structure. The final 4D mipmap may be of lowerresolution than the initial 4D mipmap used by the rasterizer. To savebandwidth, the higher resolution 4D mipmap may on-the-fly bereconstructed from the lower resolution 4D mipmap, using interpolation.

Selecting the 3D Mipmap Level for 4D Mipmap Reconstruction

Since applications usually deliver textures only in standard 3D mipmaprepresentation, in a preferred embodiment for any texture map whichprovided the rasterization grid on-the-fly 4D mipmap reconstruction froma 3D mipmap is applied. The texel shader acts then as a 4D mipmapreconstructor. As described above, the rasterizer maintains separatehorizontal and vertical mipmap levels, which together form a 4D mipmaplevel index. In the 4D mipmap texel reconstruction process, therasterizer according to the invention provides for each rasterizationgrid position the u and v coordinates for the texel that needs to befetched, along with the vertical magnification factor (or even the final4D mipmap levels (mml_(u),fmml_(v))) for that texel. The 4Dreconstruction unit needs to determine a 3D mipmap level mml from whichto fetch texels which can be filtered to provide a sample at therequested coordinates (u,v). A 3D mipmap level mml corresponds to 4Dmipmap level (mml, mml), and these are drawn as the shaded squares inFIGS. 3, 6 and 7. From these, the samples in the off-diagonal mipmaplevels need to be generated. The figures show how a texel for 4D mipmaplevel (3,0) can be generated from 3D mipmap level 3 using magnification(FIG. 6), or from 3D mipmap level 0 using minification (FIG. 3), or froma level in between (FIG. 7). Given a (possibly adjusted as describedabove) 4D mipmap level (mml_(u),fmml_(v)) provided by the rasteriser fora texel sample, a function is required which determines the 3D mipmaplevel mml to be used to address the texture memory. The following threealternatives are described here in more detail, providing differentquality/performance trade-offs.

In a low quality setting, as illustrated in FIG. 6, the 3D mipmap levelcorresponding to the coarsest of the two 4D mipmap levels is chosen.This level is then magnified in the other direction (yielding vaguenessin that direction) to arrive at the 4D mipmap level texels. Themagnification is preferably obtained using linear interpolation or, ifso desired, higher order filters. The advantage is that a minimum amountof 3D texels is fetched, and texture caching can be employed to re-usetexture samples for generation of subsequent 4D mipmap level texels(since many 4D mipmap samples are generated between the same 3D mipmaplevel samples). The 3D mipmap level selection formula corresponding tothis alternative is:mml=MAX(mml _(u) ,fmml _(v)).

In a high quality setting, as illustrated in FIG. 3, the 3D mipmap levelcorresponding to the finest of the two 4D mipmap levels is chosen. Thislevel is then minified in the other direction to arrive at the 4D mipmaplevel texels. The advantage is that maximum quality and sharpness ismaintained. But this comes at the cost of possibly many texel fetchesfor each 4D mipmap level texel that needs to be generated. The generatedtexel is preferably the unweighted average of the fetched texels (higherorder filters may be used as well). The 3D mipmap level selectionformula corresponding to this alternative is:mml=MIN(mml _(u) , fmml _(v)).

In a medium quality setting, as illustrated in FIG. 7, a compromisebetween the low and high quality setting is chosen. In a preferredembodiment, this setting is controlled by a parameter a (maximumanisotropy level) which allows for some (expensive) minification (aswith the high quality setting), but at most a mipmap levels. This meansthat at most 2^(a) texels are combined in one direction. Any largerdifference between the horizontal and vertical mipmap levels is bridgedusing an approach similar to the low quality setting. This option isillustrated in FIG. 7 for the preferred setting a=2. Setting a to 0,results in the low quality setting, while setting a very large resultsin the high quality setting. The 3D mipmap level selection formulacorresponding to this alternative is:mml=MAX(MAX(mml _(u) , fmml _(v))−a, MIN(mml _(u) ,fmml _(v))).

The lower quality settings (allowing only very limited anisotropy level)may result in artefacts, as is shown in FIG. 8. Textures which areviewed from an angle become blurry (as with traditional non-anisotropicfiltering), but since we discretely switch between mipmap levels whenusing bilinear texture filtering, jumps in blurriness become visible.Because of the per-texel 3D mipmap level selection, these lines movealong with the movements of the viewer (they are not fixed to triangleedges). If bandwidth needs to be limited using the lower qualityoptions, something like tri-linear filtering (blending between mipmaplevels) can be incorporated to make the transitions gradual. However forthe high quality setting (or for a reasonable allowed anisotropy levelin the medium setting, say a=3) this is unwanted, because at thosesettings we have enough detail, and do not want the additional blurringcaused by trilinear filtering.

Note that the value assigned to mml by the calculations described above,has to be clipped to the range of values for which mipmap levels areavailable (to exclude for example negative values of mml) andtransformed to the integer levels used to indicate the mipmap level inmemory. This may be done in any suitable way, e.g. by rounding ortruncating. The remaining fraction is preferably used for trilinearfiltering. The moment of rounding may be chosen. Preferably, the finalvalue of mml is rounded, where mml is for example determined in one ofthe three approaches described above. If so desired also Δmml, may berounded immediately when it is determined, e.g by rounding downΔmml _(v)=└²log(∂y/∂v)┘.

FIG. 9 shows a block diagram of a computer 900, including a centralprocessing unit 910, a memory 920, a display 930, and a computergraphics system 940 according to the invention. The computer may be aconventional computer, such as a personal computer, games console orworkstation. The computer graphics system may be implemented using agraphics processor. Such a graphics processor may be operated undercontrol of a program causing the graphics processor to execute themethod according to the invention. The program may be fixedly embedded(e.g. in ROM), but may also be loaded from a background memory. In thelatter case, the program may be distributed in any suitable form, e.g.using a record carrier, such as a CD-ROM, or wired or wirelesscommunication means, such as Internet.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims. In the claims, any reference signsplaced between parentheses shall not be construed as limiting the claim.Use of the verb “comprise” and its conjugations does not exclude thepresence of elements or steps other than those stated in a claim. Thearticle “a” or “an” preceding an element does not exclude the presenceof a plurality of such elements. The invention may be implemented bymeans of hardware comprising several distinct elements, and by means ofa suitably programmed computer. In the device claim enumerating severalmeans, several of these means may be embodied by one and the same itemof hardware. The mere fact that certain measures are recited in mutuallydifferent dependent claims does not indicate that a combination of thesemeasures cannot be used to advantage.

1. A system for rendering an image for display, including: a texturememory (134) for storing texture maps in a mipmap structure; texels in atexture map being specified by a pair of u and v coordinates; arasterizer (120) operative to, for a texel (u, v), determinecorresponding initial 4D mipmap levels (mml_(u), mml_(v)); determine amagnification factor representing a magnification that occurs when thetexel is mapped to a corresponding pixel position on the display; anddetermine corresponding final 4D mipmap levels in dependence on thedetermined initial 4D mipmap levels mml_(u), mml_(v), and themagnification factor; and a texture space resampler (132) for obtainingtexture data from a texture map identified by the pair of final 4Dmipmap levels; a texture mapper (140) for mapping the obtained texturedata to corresponding pixel data defining the display image.
 2. A systemas claimed in claim 1, wherein the magnification factor represents amagnification in a vertical direction indicated by coordinate v.
 3. Asystem as claimed in claim 2, wherein the rasterizer is operative todetermine a final vertical 4D mipmap level fmml_(v) by adjusting mml_(v)to identify a lower resolution vertical 4D mipmap level if themagnification factor is less than a predetermined threshold andmaintaining the determined mml_(v) mipmap level otherwise.
 4. A systemas claimed in claim 1, wherein: the texture memory is arranged to storethe texture maps in a 4D mipmap structure; each texture map beingidentified by a pair of 4D mipmap levels; the texture space resampler isoperative to on-the-fly reconstruct at least part of a texture map of a4D mipmap identified by the pair of initial 4D mipmap levels from atexture map of a 4D mipmap in the texture memory identified by the pairof final 4D mipmap levels for use by the rasterizer.
 5. A system asclaimed in claim 1, wherein: the texture memory is arranged to store thetexture maps in a 3D mipmap structure; each texture map being identifiedby a respective 3D mipmap level mml; the texture space resampler isoperative to on-the-fly reconstruct at least part of a texture map of anidentified 4D mipmap from an associated 3D mipmap with level mml in thetexture memory.
 6. A system as claimed in claim 3, wherein the 3D mipmaplevel mml of the associated 3D mipmap is given by MAX(mml_(u),fmml_(v)).
 7. A system as claimed in claim 4, wherein the 3D mipmaplevel mml of the associated 3D mipmap is given by MIN(mml_(u),fmml_(v)).
 8. A system as claimed in claim 4, wherein the 3D mipmaplevel mml of the associated 3D mipmap is determined in dependence on apredetermined maximum anisotropy level a.
 9. A system as claimed inclaim 8, wherein the 3D mipmap level mml of the associated 3D mipmap isgiven by MAX(MAX(mml_(u), fmml_(v))−a, MIN(mml_(u), fmml_(v)) )
 10. Acomputer including a central processing unit, a memory, a display, and asystem as claimed in claim
 1. 11. A method of rendering an image fordisplay, including: storing texture maps in a mipmap structure; texelsin a texture map being specified by a pair of u and v coordinates; in arasterization operation determining, for a texel (u, v): correspondinginitial 4D mipmap levels (mml_(u), mml_(v)); a magnification factorrepresenting a magnification that occurs when the texel is mapped to acorresponding pixel position on the display; and corresponding final 4Dmipmap levels in dependence on the determined initial 4D mipmap levelsmml_(u), mml_(v), and the magnification factor; in a texture spaceresampling operation, obtaining texture data for a texture mapidentified by the final 4D mipmap levels; and mapping the obtainedtexture data to corresponding pixel data defining the display image. 12.A computer program operative to cause a processor to perform the methodof claim 11.